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ABSTRACT 



This paper provides an overview of the performance 
indicator-based accountability and funding mechanism implemented in the 
higher education system of Alberta, Canada. The paper defines the terms 
accountability and regulation, examines the use of performance indicators to 
demonstrate accountability, and explains how performance indicator-based 
accountability and regulatory mechanisms are used as policy instruments. The 
paper also discusses traditional approaches to resource allocation and recent 
developments in resource allocation, including performance-based funding. The 
paper outlines the pressures placed on governments by globalization, the 
right-wing political ideologies that give these pressures political voices, 
the resultant change in the role of government, and the development of 
academic capitalism. The paper notes that government policy has induced 
academic capitalism in Alberta's higher education system and describes 
Alberta's performance-based funding mechanism. The paper concludes by 
outlining some criticisms of the performance indicators used in Alberta's 
funding mechanism and discussing the implications of the performance-based 
funding mechanism in the province. (Contains 85 references.) (MDM) 
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1. Introduction 

Like many jurisdictions, Alberta (Canada) has implemented a performance indicator-based accountability 
and funding mechanism in its higher education system. Sections 2, 3 and 4 provide an introduction to the 
use of Pl-based mechanisms for accountability and regulatory purposes and situate them in the broader 
policy enviromnent. Sections 5 and 6 discuss and assess Alberta’s performance-based funding mechanisms. 

2. Accountability, regulation and performance indicators in higher education 

This section outlines how performance indicators (Pis) are related to accountability and regulation. Section 

2. 1 defines accountability and regulation. Section 2.2 examines tlie use of performance indicators to 
demonstrate accountability. Section 2.3 explains how performance indicator-based accountability and 
regulatory mechanisms act as policy instruments. Subsequently, Section 3 discusses how accountability 
reports are linked to funding to create de facto regulation. 

2. 1 Accountability and regulation 

Accountability is tliat which is exchanged for autonomy in an authority relationship (McDormell, 1994; 
Neave, 1980). Being accountable entails providing a report of one’s performance and being responsible for 
that performance (Wagner, 1989; Ewell, 1987). To be held accountable, one must be causally responsible 
(i.e., one’s performance must result from an act committed or omitted) and one must be expectationally 
responsible (i.e., one’s performance must be judged against a reasonable standard in light of one’s mission 
and circumstances). Those to whom one is accountable must possess authority (i.e., have a valid interest in 
one’s performance and tlie riglit to judge it), the origins of which may be contractual, statutory or moral 
(Neave, 1982). Autonomy is the freedom to act and can be substantive (i.e., having the freedom to pursue 
academic matters witliout interference) or procedural (i.e., having the freedom to establish administrative, 
budgetary and operational policy and procedures). Kells (1992) outlines three approaches to accountability: 

1. Goal achievement — focuses on comparing internally set goals with outputs or outcomes. 

2. Process and environment — ^focuses on institution’s internal envirorunent or processes. 

3. Compliance with standards — focuses upon comparing outcomes with externally set standards. 

Tlie growing trend towards basing resource allocation upon accountability data results in regulation (Kells, 
1992). Regulation involves examining a performance and acting to maintain or change it. Regulatory 
mechanisms are also being linked witli other policies in a mutually reinforcing marmer to advance a 
broader agenda tliat is outlined in Section 4 (Cilbulka and Derlin, 1998). 

2.2 Performance indicators as evaluative tools 

Demonstrating accountability entails evaluation. Because direct evaluation may be intrusive and costly (as 
well as potentially affecting tlie performance being evaluated), evaluation often occurs by proxy (Banta, 
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1993). In these cases, characteristics, events and (most commonly) outcomes that are believed correlated 
with tlie desired performance are used as performance indicators. Cave, Hanney, Henkel and Kogan (1997) 
classify indicators as simple (neutral descriptions), general (data unrelated to goals), and performance 
(possessing a point of reference for comparison). Numeric perfonnance indicators dominate government 
introduced accountabiUty mechanisms and operationalize concepts such as quality by specifying 
measurable evidence of goal completion (Dochy, Segers and Wijnen, 1990; Jones, 1982). As evaluative 
tools. Pis require goal agreement (Richardson, 1994; Wagner, 19889). Because Pis measure institutional 
performance, institutions must be able to affect their PI scores (Sizer, Spee and Bormans, 1992). Kaufmann 
(1988) identifies five organizational elements to which indicators can be applied: 

1. Inputs are raw materials (e.g., resources, policies, communal characteristics). 

2. Processes are tlie ways inputs become products, outputs and outcomes (e.g., teaching, learning). 

3. Products are results tliat are fed back into the system to become outputs and outcomes (e.g., courses 
tliat eventually result in degrees awarded). 

4. Outputs are tlie aggregate products of a system (e.g., degrees awarded, papers pubUshed). 

5. Outcomes are tlie effects of outputs in society (e.g., employment rates, life expectancy, democracy). 

Kaufmann views organizational success in terms of goal attainment and, therefore, inputs and processes are 
means by which to create products, outputs and outcomes. Performance indicators have several uses: 

1. Informing higher education planning. Accountability systems defined by institutional governors tend 
to be closely tied to planning (Ruppert, 1995). Tracking and projecting trends can provide the 
warning and infonnation necessary to plan effective interventions and improve efficiency in order to 
properly steward tlie public’s investment (Nedwek and Neal, 1994). Tliis is highly dependent on 
identifying patterns of change and leading indicators (Freeman, 1995). 

2. Improving higher education practice. Determining relationships between inputs, process and outputs 
increases both educators’ understanding of their work and the quality of educational products, 
outputs and outcomes (Sizer et al., 1992). The complexity of social systems may impede accurate 
causal modeling (Ewell and Jones, 1994). 

3. Monitoring the outcomes of higher education. Reporting on the contribution of higher education to 
identifiable outcomes empliasizes the return on investment (Banta, Rudolph, Van Dyke and Carter, 
1996). This application of Pis informs consumer choices (Cave et al., 1997). 

Pollster and Newson (1998) criticize Pis as conceptual technologies that facilitate the entry of non- 
academic agendas into higher education. Peters (1992a) notes tliat using Pis involves judgment in selecting 
indicators and setting reference points and, therefore. Pis are not a value-neutral technology. By making 
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visible the most and least efifective work of academics (based upon a cost-benefit analysis), Pis make it 
possible to differentially reward faculty such that accountability becomes regulation. 

Pis are subject to a host of technical criticisms as well. Pis are drawn from industrial input-output analysis 
(Emberley, 1996). This approach assumes that inputs are uniform and passive and that outputs are 
standardized tlius making possible comparisons to find the best (i.e., ma.\imally effective and efficient) way 
to transform inputs into outputs. If inputs and outputs are variable, then a common metric by which to 
measure is necessary to make valid comparisons between processes. Education provides neither uniform 
inputs and outputs nor common metrics. Students and instructors (i.e., inputs) are neitlier passive nor 
uniform. Graduates (i.e., outputs) are also umque in the degree to which tlieir knowledge, skills and 
attitudes can be developed and for what ends and manifest tliemselves over an extended period of time. The 
process (involving introspection, synthesis and integration) is similarly variable according to students’ 
experiences, aptitudes and motivation and is thus resistant to the imposition of a common metric. Nedwek 
and Neal (1994) argue tliat the pervasion of an input-output (i.e., mechanistic) model of education neither 
fully models tlie educational process nor captures all educational outcomes. Also of concern is that Pis are 
conceptually better suited to measuring economic outcomes; adopting this approach conceptually closes the 
door to measuring social and cultural outcomes which are generally measured qualitatively. 

2.3 Accountability mechanisms as policy instruments 

Accountability mechanisms are policy instruments in that they translate policy goals into actions by 
propelling individuals to act when otherwise tliey could not or would not. Four types of instruments emerge 
from the literature (McDonnell, 1994; Schneider and Ingram 1990): 

1 . A uthority-based instruments grant pennission, require action or change the distribution of authority. 

2. Incentive-based instruments use inducements, sanctions, charges or force to encourage action. 

3. Capacity-building instruments invest in intellectual, material or human resources to enable action. 

4. Hortatory instruments signal goals are of high priority to propel action. 

Incentive-based instruments are generally short-term and designed to attain specific objectives while 
capacity-building instruments are generally long-term and designed to create the potential for gains. Hybrid 
mechanisms (e.g., combining incentive and autliority components) may increase instruments’ effectiveness. 

3. Resource allocation and regulation 

Allocating resources based upon accountability data creates de facto regulation. Section 3. 1 outlines 
traditional approaches to resource allocation. Recent developments in resource allocation such as 
performance-based funding are outlined in Section 3.2. Subsequently, Section 4 outlines the broader policy 
agenda tliat is driving tlie use of performance indicator-based accountability and regulatory mechanisms. 
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3. 1 Traditional approaches to resource allocation 

Linking resource allocation to funding creates regulation. Institutional funding has traditionally been 
distributed on an incremental or formula basis (Massy, 1996; Camthers, Marks and Walker, 1994). 
Incremental budgeting changes basic operating and capital budgets annually (Layzell and Caruthers, 1995; 
Epper, 1994). Tliis perpetuates resource distribution patterns and is criticized as irrational as well as 
encouraging budget maximizing behaviour by bureaucrats (McKenzie, 1997; Osbourne and Gaebler, 1992). 
Formula-based budgeting ties allocations to specific criteria (e.g., $6200 per full-time equivalent student) 
and is designed to ensure funding adequacy, equity and stability (Elmore, Abelmann and Fuhrman, 1996; 
Caruthers et al., 1994). Critics believe formula funding creates inter-institutional competition for the basis 
of allocations (e.g., students, programs, physical plants, etc.) but provides no incentive for institutions to 
improve functioning (Albright and Gilleland, 1994; Sells, 1994). Both incremental and formula funding 
delegate allocation decisions to institutional governors. 

3.2 Recent developments in resource allocation 

Performance-based and incentive funding are new methods of resource allocation. Performance-based 
funding sets measurable outcomes, ties future funding to outcome attainment and thereby introduces 
market-like competition between institutions that is structured by government (Serban, 1997; Layzell and 
Carutliers, 1995; Peterson, Erwin and Wilson, 1977). Incentive funding makes available additional funds 
contingent upon institutions engaging in specific activity (Epper, 1994). The structure and criterion of 
incentive and perfonnance-based funding allow governments (rather than institutions) to operationalize 
concepts such as quality and impose agreement on values and purposes (Holland and Berdahl, 1990; Carter, 
1989). Performance and incentive funding tend to function on the edges of, or in addition to, base 
allocations because substantial planning difficulties and intense inter-institutional competition result from 
large annual redistributions (Ashworth, 1994). Allocations must be large enough to gamer credibility from 
legislators, tlie public and institutions (Banta, 1993). Underlying these approaches is the belief that 
institutions should and must be forced to re-examine basic spending patterns (Epper, 1994) and that altering 
institutional spending patterns is desirable (Massy and Hulfactor, 1993; Brown and Wolf, 1993). This 
ignores tliat funding patterns represent negotiated solutions that ensure multiple and often-conflicting 
objectives are achieved (Camtliers et al., 1994; Carter, 1989). 

Fully appreciating tlie implications of the growing use of Pis and performance-based funding mechanisms 
(PBFMs) requires an understanding of the broad structural changes occurring within Canadian society. 
Section 4 outlines tlie effect of economic globalization and right-wing political ideologies on the role of 
govermnent and its approach to higlier education. 
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4. Academic capitalism and the evaluative state 

Tlie growing use of Pis to drive regulatory mechanisms is part of a policy agenda to align higher education 
witli market needs. Section 4. 1 outlines the pressures placed on governments by economic globalization. 
Section 4.2 outlines tire right-wing political ideologies that give these pressures political voice. Section 4.3 
outlines the resultant change in tire role of government. In higher education, this has induced academic 
capitalism as outlined in Section 4.4. Subsequently, Section 5 outlines Alberta’s use of a performance 
indicator-based accountability and regulatory mechamsm. 

4. 1 Economic globalization 

The globalization thesis posits that maintaining economic stability requires governments to attract 
investment capital and that the new, transnational nature of capital pressures governments to reduce tax- 
funded public services so as to increase investors’ returns (Castles, 1996). Teeple (1995) explains the 
Keynesian Welfare State (KWS) as a result of capitalists’ need for social reform to maintain production. 
Capitalism’s tendency to increase economic disparity is symptomatic of a fundamental conflict between 
democracy and capitalism: as a distributive system for social goods, democracy’s distributive criteria are 
citizenship wliile capitalism’s distributive criteria are the ability to pay. The KWS’s large public sector 
mitigates tliis crisis tendency (Jessop, 1993). Transnational capital no longer requires KWS reforms 
because the interests of nation-states and corporations have been uncoupled: if production falters, capital 
relocates (Greider, 1997). Tlie role of the state becomes to adjust domestic economies to meet the needs of 
globalization (Dominelli and Hoogvelt, 1996). 

Dudley (1998; Brown and Lauder, 1996) categorize educational policy responses to globalization as a neo- 
Fordist or post-Fordist. A neo-Fordist approach emphasizes labour-market flexibility as a means to reduce 
production costs and increase competitiveness. Efficiency is sought through deregulation, managerialism 
and privatization. Higher education is focused on meeting the training demands of industry (Neave, 1980). 
Tlie post-Fordist approach emphasizes developing a highly and multi-skilled workforce to stimulate and 
attract high wage jobs. Tlie state’s role is to develop human capital and guide growth through targeted 
investments (Newson, 1994). The adoption of a neo-Fordist approach in Alberta reflects a political 
compromise bridging an ideological division within the Progressive Conservative Party. 

4.2 Ideologies and the rightist alliance 

Ideologies are collections of assumptions, values and expectations used to make sense of the world and 
guide beliefs about economic production and distribution (Dolbeare and Medcalf, 1988). Liberalism 
dominates Canadian politics and assumes tliat self-interest motivates individuals and rewards reflect merit 
(Gibbins and Youngman, 1996). Classical Liberalism emphasizes negative freedom (i.e., equality of right 
and freedom from constraint) while Reform Liberalism emphasizes positive freedom (i.e., equality of 
opportunity and freedom to act). Teeple (1995) attributes the decline of the KWS’s Reform Liberalism to 
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economic stagnation, growing business influence due to globalization and a backlash against social change. 
This has opened the door for a resurgence of Classical Liberahsm as well as the emergence of 
Neoliberalism, Neoconservatism and the New Right. 

Neoliberals describe themselves as pragmatic and use incentives, assistance and infrastructure 
improvement to generate prosperity and help the disadvantaged increase their self-sufficiency (Pratt, 1998; 
Gibbins and Youngman, 1996). Neoconservatives are reacting against growing egalitarianism (which 
threatens order and autliority), desire a more authoritative state and oppose govermnent attempts to acliieve 
greater social equality. The New Right is a populist movement — combining religious fundamentalists and 
tlie disaffected lower middle class — that desires radical minimize govermnent and advanced traditional 
values. Apple (1998) posits an alliance among Neoliberals, Neoconservatives, Classical Liberals and the 
New Right tliat uses tlie market to determine what and how much is produced, how production is organized 
and how goods are distributed (Fellows, Flanagan, Shedd and Waud, 1993). Within this alliance is 
substantial disagreement regarding social change and the role of government. Neoliberals and Classical 
Liberals advocate the use of markets to generate social change instead of state intervention. This conflicts 
with the Neoconservative and New Right belief that an authoritative state is necessary to arrest societal, 
moral degeneration. Tliis conflict has given rise to the evaluative state. 

4.3 The evaluative state and accountability mechanisms 

The evaluative state (also called contract govermnent) devolves goal attainment responsibility to 
individuals and institutions while government retains control of resource allocation (Neave, 1988). 
Traditionally, governments set goals and then allocated resources. Evaluation was a priori in that it 
assumed goal attaimnent would occur because resources were provided (Elmore, 1 979/80). Tliis approach 
corresponds witli incremental and formula-based budgeting. The evaluative state relies on a posteriori 
evaluation: resources are allocated based upon prior performance to encourage goal attainment. This 
approach is consistent ivitli incentive and performance-based funding and recognizes that policymakers 
may be better able to control implementation through outcomes assessment. The evaluative state allows 
government to: create a market-like environment ivitli quasi-independent agencies; measure tlieir outputs; 
and tie tlie value of outputs to rewards. Accountability mechanisms are necessary in the market-like 
environment to prevent dishonesty motivated by the pursuit of maximal personal utility. The measurement 
of performance to evaluate outcomes, increase productivity and provide consumer information naturally 
flows from tlie market-based approach (Peters, 1992b). Tlie market-like insulates government from direct 
responsibility for inequitable outcomes: outcomes stem from tlie actions of independent agencies and the 
impersonal forces of the market. In liiglier education, the evaluative state has induced academic capitalism. 
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4.4 The emergence of academic capitalism 

The emergence of an evaluative state in response to economic and ideological pressures has induced 
academic capitalism in higher education. Academic capitalism exists when institutions and faculty 
members engage in market (i.e., for profit activities) and market-like (i.e., competing for funding) 
behaviours such as seeking grants, launching spin-off companies, building endowments, raising tuition and 
entering into business-education partnerships (Slaughter and Leslie, 1997). 

Resource dependence theory says tliat organizations depend upon their environment for key resources and 
organizational behaviour is a response to the actions of external agents who control organizational 
resources (Pfeffer and Salancik, 1978; Pfeffer, 1992). Substantial changes in resource availability 
destabilize organizations and result in organizational adaptation to ensure survival. Making three changes 
to liigher educaUon funding (Slaughter and Leslie, 1997; Newson, 1994; Buchbinder and Newson, 1990) 
has induced academic capitalism: 

1. decreasing public transfers of undesignated (i.e., block) funding; 

2. increasing use by government of designated flmding (i.e., funding requiring of institutions specific 
performances); and 

3. increasing reliance on external funders who (naturally) stipulate the use of their flmds. 

Decreasing tlie amount of undesignated public funding forces institutions to seek new (generally non- 
govenunental) sources of funding and, therefore align their activities with the needs of the market. Further, 
increasing tlie use of designated flmding allows governments and private funders to direct research, 
curricular and administrative decisions (Furstenbach, 1993). Together, these changes decrease institutional 
autonomy by encouraging a growing alignment between the needs of the maiket and the activities of post- 
secondary institutions. Accountability and regulatory mechanisms are part of the growing trend towards 
using designated funding “envelopes”. Accountability mechanisms are also used to legitimate the 
introduction of academic capitalism. Specifically, they: 

• focus attention on outputs and outcomes thereby shifting attention to operational issues and away 
from tlie impact of policy decisions on equity; 

• legitimate tlie introduction of market beliaviour to academe by imposing the excliange relationship 
of the market economy on liiglier education 

• offload responsibility onto institutions for delivering services while government has a solely 
evaluative role; 

• reinforce tlie market dynamic by tying funding to performance, thereby intensifying the market 
forces institutions are already subjected to; and 
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• focus attention on easily quantifiable aspects of education such as efficiency while ignoring more 
difficult to quantify aspects such as quality. 

The introduction of academic capitalism has a number of implications for higher education systems. 
Perhaps most importantly, realigning higher education’s activities to meet market needs allow capitalists 
(i.e., employers and investors) to present their interests as the interests of society (Harvey and Knight, 

1996). This harnesses higher education to serve the needs of the wealthy and perpetuates the economic 
disparities generated by market systems. There are other implications of academic capitalism, some of 
which reinforce tliis sliift in power and control; 

• Institutional activities become externally driven as fiscal pressure requires administrators to be more 
attentive to the needs of governments and business. 

• New resource allocation structures, including greater sophistication and competitive bidding, change 
the degree of control exercised by institutions over internal activities. 

• Documentary decision-making (e.g., PBFMs) begins to usurp the role of orally contested decision- 
making tlu-ough Boards of Governors and academic councils. This transfers power to those who 
construct and operate documentary systems. 

• Tire greatest barrier to having market-oriented institutions is the long-term employment 
arrangements institutions have with their academic staff. Greater reliance on part-time and limited- 
term instructional s taff increases institutions’ ability to meet the needs of the market. 

• Tire commodification of education results in students acting increasingly like consumers. This 
threatened tlie delicate balance higher education maintains between its service and coercive 
mandates. As institutions increasingly cater to students’ desires, the credibility of credentials the 
value of which lie in their certification that students are familiar with and competent in the canon 
and methods of a field — may suffer. 

Newson (1994) notes tliat the belief tliat institutions will survive the introduction of academic capitalism 
and be increasingly able to insulate core areas through market activity is difficult to substantiate. 
Reductions in government funding more tlian offset additional corporate funding and corporate funding 
comes with many strings attached. Institutions are not free agents entering into exchange relationships 
because they are resource poor. Partnersliips witli corporations requires increasing managerialism which in 
turn increased tlie influence of external groups over teaching and research. This alters the knowledge 
creation purpose (towards profit), process (towards consumption) and values (towards market-based). 

5. Accountability and performance-based funding in Alberta 

Tliis component of the paper outlines tlie introduction of Alberta’s performance indicator-based 
accountability and funding mechanism. Section 5.1 outlines how government policy has induced academic 
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capitalism in Alberta’s higher education system. Section 5.2 describes Alberta’s performance-based 
funding mechanism (PBFM). Subsequently, Section 6 presents an analysis of the mechanism, its 
implications for educators and some speculation as to its future development. 

5. 1 Academic capitalism in A Iberta 's higher education system 

Alberta’s public higher education system enrolls approximately 123,000 students in 4 universities, 2 
technical institutes, 15 colleges and 4 rehgiously affiliated, not-for-profit university colleges (AECD, 
1998a). Government funding of approximately $788 million was delivered in 1998 through the Department 
of Advanced Education and Career Development in two ways. A base grant (a combination of operating 
grants fixed at 1991/92 levels and capital funds fixed at 1986/87 levels) is provided each year. A series of 
funding envelopes were introduced in 1994 and allocated funding on a competitive basis to encourage 
institutions to achieve government goals (Treasury, 1998). Institutions must generate and submit for 
approval three-year business plans and armual reports. 

Tire White Paper New directions for adult learning in Alberta (AECD, 1994) signaled the official 
introduction of academic capitalism to Alberta’s higher education system (Bametson, 1999a). While New 
directions noted that “(t)he adult learning system exists to serve the social, cultural and economic goals of 
the learner and tire community” (AECD, 1994, p.7), its policy initiatives focus on increasing the ahgnment 
of the higher education system with the market by continuing to lower per-student government 
expenditures and eroding institutional autonomy. 

The three-year 21% reduction to institutional base grants that began in 1994 continued a long-term decline 
in public funding (AECD, 1994). When the 21% reduction is combined with rising enrollments and 
sluggish funding growth tlvough the 1980s, per-student, real dollar (1997=100) government grant-based 
revenue has fallen from $14,551 in 1982/83 to $7968 in 1997/98— a reduction of 45.3% over 15 years 
(Sliillington, 1998). Altliougli tuition lias increased, tlie decline in tuition- and grant-based revenue still 
totals 37%. Additional pressure will be exerted on Alberta’s higlier education system by an expected 
increase in enrollment between 23,000 and 37,000 students between 1997 and 2005 (AECD, 1997a). 
Altliough some additional funding has been committed to manage this growth, projections suggest that per- 
student funding levels will continue to decline (ACIFA and CAFA, 1998). This pressures institutions to 
enter into partnersliips with tlie private sector and thereby surrender some of their autonomy. 

The introduction of funding envelopes as further erodes autonomy by allowing government to stipulate 
goals. Institutions’ proposals or performances are compared to criteria and allocations are made on a 
competitive basis. Envelope funding is projected to increase from 5% of government transfers to 
institutions in 1996/97 to 17% in 2000/01. Each funding envelope is designed to elicit or encourage a 
specific performance from institutions and that performance is in addition to their existing mandate. These 
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envelopes serve to align institutional activities with market needs by demanding public-private 
partnerships, and by favouring market-proximate disciplines (Bametson, 1999a). The Performance 
Envelope mandated tlie creation of a performance-based funding mechanism (PBFM) which is described 
below. 

5.2 Alberta 's Performance Envelope and Performance-based Funding Mechanism 
New directions mandated tlie creation of an accountability reporting framework to outline tlie outputs and 
outcomes of Alberta’s liigher education system (AECD, 1994). Criticism of existing accountability 
mechanisms included a focus on inputs and processes, a lack of systematic use, and inconsistent data 
definitions tliat impede inter-institutional comparisons (AECD, 1995a). A series of 76 Pis were developed 
in consultation with institutions (AECD, 1996a, 1996b). A second White Paper-mandated goal was the 
creation of “a new funding mechanism that rewards an institution’s performance in providing accessibility, 
quality and relevance” (AECD, 1994a, p. 15). This resulted in the development of a performance-based 
funding mechanism to distribute $15 million annually from the Performance Envelope. 

Tlie PBFM is driven by nine of the framework’s 76 indicators (AECD, 1997b); five indicators are used by 
all institutions (tlie learning component) while four indicators are for research universities only (the 
research component). A two-year pilot of tlie mechamsm began in 1996/97 with the first performance 
awards announced in July, 1997. Tlie performance awards are partly funded by a 0.5% clawback from 
institution’ s base grants as well as by $ 1 5 million in new government funding each year. A review of the 
mechanism is pending but government budget documents suggests that in 2000/01, there will be no new 
money added to tlie Performance Envelope although the value of performance awards will increase 
(Treasury, 1999; AECD, 1999). This indicates that the project will begin redistributing funding within the 
(closed) system. 

Tlie PBFM takes an institution’s numeric score on an indicator (e.g., percentage of graduates employed) 
and plots it on a linear scale (e.g., 0-100%). Benchmarks divide the scale into a series of performance 
corridors (e.g., 60-69%, 70-79%, 80-89%, >89%); all institutions falling within a corridor are assigned the 
same number of points for that indicator. Tlie points assigned for performance on each of the five learning 
component indicators are tallied and tliat score constitutes overall performance for funding award (AECD, 
1996c). Research universities engage in a similar process with the research component indicators. 
University research results are weighted based upon the amount of institutional funding that is directed to 
research. Tlie structure of all of the indicators is outlined in Tables 6. 1 and 6.2. 

Each institution’s data is drawn from its four largest programs (AECD, 1996c). The level of aggregation at 
which performance indicators are compared reflects a compromise. Comparing performance at the level of 
program (e.g., undergraduate political science programs) is problematic because of small sample sizes. 




10 

12 



variations in program methods and nature of programs, and annual variations (AECD, 1996d). Comparing 
clusters of programs (e.g., all undergraduates arts and science programs) eliminates these concerns at the 



Table 6.1 Learning component indicators 

Employment rate: Percentage of graduate-survey respondents employed within a specified period 
following program completion. 

Points: 0 15 20 25 30 

\ \ \ \ 

Benchmarks: 60% 70% 80% 90% 



Graduate satisfaction with overall quality: Percentage of graduate-survey respondents fully/somewhat 
satisfied witli overall educational quality. 

Points: 0 15 20 25 30 

\ \ \ \ 

70% 80% 90% 95% 

Percentage change in full-load equivalent enrollment from one period to the next. 

0 20 25 30 

\ 1 1 

Benclimarks: Urban -2% 0% +4% 

Rural -5% 0% +4% 

Administrative expenditures: Administration as a percentage of total expenditures less ancillary 
expenditures. 

Points: 0 3 4 5 



Benchmarks: > 3500 students 11% 7% 5% 

^ 3500 students 12% 8% 6% 

Enter|)rise revenue: Revenues less all government grants, tuition fees under policy, sponsored research 
(universities only), ancillary services and earned capital contributions as a percentage of Advanced 
Education and Career Development grants. 

Points: 13 4 5 



Benchmarks: Urban 20% 35% 50% 

Rural 10% 25% 40% 



Benchmarks: 
Credit FLE: 
Points: 



cost of specificity. Scores are additive (i.e., represent the weighted sum of the individual Benchmarks 
determined at tlie program level). 
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The learning component’s five indicators are broken down into three categories based upon New 
direction *s key goals of responsiveness, accessibility and affordability (AECD, 1994). Institutional 
responsiveness to the needs of learners and to provincial social, economic and cultural needs is assessed by 
examining the employment rates of graduates and graduates’ satisfaction with the quality of their 
educational experience. Institutional progress towards higher levels of accessibility (i.e., increasing the 
number of student spaces) is indicated by examining changes in full-load equivalent (FLE) enrollment 
based upon a tliree-year rolling average. This indicator is adjusted for institutional location and recognizes 
that urban institutions have historically been better able to maintain stable enrollments because of a larger 
population base (AECD, 1996d). Institutions’ success at maintaining affordability (i.e., providing quality 
learning opportunities to the greatest number of Albertans at a reasonable cost to the learner and taxpayer) 
is indicated by examining administrative exTenditures and outside revenue generation. 



Tabic 6.2 Research component indicators 

Council monetary awards: National peer group rank in terms of council awards per full-time faculty 
member. 



Points for achievement: 


0 


17 


25 


Points for improvement: 


0 


10 


15 


Benchmarks: 


1 

Bottom third 


Second third 


1 

Top third 


Citation impact: National peer group rank in terms of number of citations per research publication. 


Points for achievement: 


0 


17 


25 


Points for improvement: 


0 


10 


15 


Benchmarks: 


1 

Bottom third 


Second third 


1 

Top third 


Community and industry support: National peer group rank in terms of community and industrial 


funding for sponsored research per fiill-time faculty member. 




Points 


0 


17 


25 


Benchmarks: 


1 

Bottom tliird 


Second third 


1 

Top third 


Research enterprise: National peer group rank in terms of sponsored research revenues as a percent of 


AECD grants. 








Points 


0 


3 


5 


Benchmarks: 


1 

Bottom tltird 


Second third 


1 

Top third 
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The research component has four indicators (AECD, 1997b). Council success rates identifies national 
granting council awards (MRC, NSERC and SSHRC) per full-time faculty member. This is done for peer 
institutions across Canada. Citation impact is the expressed ratio of citations to published papers. The 
Institute for Scientific Information produces a database of summary publications and citation statistics that 
reflect research performance in the sciences and social sciences for Canadian universities. This database 
includes citations for 6000 peer-reviewed journals. Community- and industry-sponsored research per full- 
time faculty member is derived from Statistics Canada and die Canadian Association for University 
Business Officers data. Research enterprise is the total sponsored research revenues generated and is 
expressed a proportion of tlie govenunent grant. This indicator is meant to capture the degree of “leverage” 
an institution lias established through its operations grant by raising research funding. 

During tlie two-year pilot, $15 million was allocated annually from the Performance Envelope. In 1997, 
tliis money was distributed in two ways. First, each institutions received a system-wide award of 1% of its 
operating grants to reward system-wide improvements in productivity. From this 1%, 0.5% was clawed 
back to partially finance tlie Performance Envelope. Second, institutions were awarded based upon their 
performance: 8 (of 21) institutions received an additional award of 1.5% based upon their performance, 9 
institutions received an additional .75% and 4 institutions received no performance-based funding (AECD, 
1997d). Tlie percent of operating grants received as a performance award in 1998 decreased slightly to 
1.26% and 0.63%) because, while institutions’ performance improved, the total funding for performance 
awards available remained fixed at $15 million (AECD, 1998b). These awards accumulate over time (i.e., 
are added to institutions’ revenues the next year) although they remain as part of the funding envelope line 
item in tlie provincial budget (Treasury, 1998). 

6. Criticisms of performance-based funding in Alberta 

Altliough tlie actual amount of funding distributed througli the Performance Envelope is small, this 
envelope has some significant implications. Section 6. 1 outlines some criticisms of the Pis used. Section 
6.2 tlien describes some of the implications of the PBFM. Section 6.3 speculates as to the future of 
Alberta’s performance indicator-based accountabihty and funding mechanism. 

6.1 Criticisms 

A number of technical criticisms can be directed at the specific indicators used in Alberta’s performance- 
based funding mechanism. This section draws heavily from Bametson (1999b). 

Accessibility indicator— Tlie Credit FLE indicator was a near perfect predictor of overall performance on 
tlie learning component for community colleges, universities and teclinical institutes in 1997 and 1998 thus 
disproportionately influenced overall institutional performance (AECD, 1997d, 1998b). Based on these 
results, institutions wisliing to receive the maximum performance award must annually increase their 
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enrollments by more than 4%. These enrollment increases compound over time in a manner similar to 
interest and enrollment grows at a faster pace than the 1-2.5% of operating grants awarded as performance 
funding that tire accompanies the additional students (Bametson, 1997a). This deficiency may be offset by 
additional funding from a separate funding envelope focusing on increasing the number of student spaces, 
but the overall decline in per-student funding is consistent with the government’s commitment to increasing 
institutional productivity (Duncan, 1997). By increasingly off-loading the cost of carrying additional 
students onto institutions, the PBFM encourages institutions to decrease their enrollments by relatively 
better rewarding a decrease. This suggests that the PBFM’s reward structure will not accomplish the 
govenunent’s goal of increasing the number of student spaces available. This will be mitigated by the 
Access Fund (a funding envelope used by government to expand specific programs). In combination, the 
Access Fund and tire PBFM actually increase the government’s leverage: there is no incentives for 
institutions to expand any programs except those which receive Access Funding. To date, the Access Fund 
has focused on creating or expanding labour-market focused programs. 

A related criticism is the disagreement within the system about the goal of the indicator, Ewell and Jones 
(1994) note tire importance of clear and mutually agreed upon benchmarks of progress. Tlrere remains 
significant resistance to increasing enrollments at less than full funding among faculty. Tire resultant 
increase in faculty-to-student ratios (exacerbating existing shifts in instructional and evaluation practices) 
are generally viewed as negative developments by those who implement and are affected by policy at the 
institutional level (CAFA, 1997; ACIFA, 1997; ACTISEC, 1997). 

Finally, using tire same indicator for all institutions (which vary from a research-intensive university with a 
medical school to a series of small agricultural colleges) may be problematic. Wlrile there is recognition of 
tire impact of geographic location, there is no consideration of the impact of program types. Programs are 
differentially expensive to expand based upon their infrastructure requirements. Adding five sections of 
introductory English composition (assuming adequate lecture space) comes at the cost of an additional 
instructor and perhaps some library resources. Adding five sections of introductory animal husbandry 
requires an instructor, additional animals and appropriate infrastructure and support for tlrem. Other 
infrastructure-intensive programs such as teclmical programs are similarly handicapped. The indicator will 
advantage institutions offering less-expensive programs . 

Responsiveness indicators — While tire PBFM is supposed to encourage responsiveness in post-secondary 
institutions to tire needs of individual learners and tire social, cultural and economic needs of the province, 
tire mechanism disregards tire social and cultiu^l needs by noting tlrat “ . . .while important, (they) have not 
been clearly articulated” (AECD, 1996b, p. 6). Tlris is a significant omission and may reflect tire inability 
of quantitative Pis to fully engage tire concept of educational quality (Sclrmitz, 1993). 





The employment-rate indicator assumes a causal relationship between institutional activity and graduate 
employment. This relationsliip is imdoubtedly confounded by both the general level of employment (which 
fluctuates significantly as a result of Alberta's resource-based economy) and the attributes/activities of the 
graduates — ^both factors that institutions have little or no control over (Carter, 1989). In the longer term, 
institutions can control these factors by closing programs with imstable enrollments. This in a politically 
unsaleable option in Alberta because of a strong tradition and expectation that many programs will be 
available in every region of the province. 

From a technical perspective, the indicator — which combines students employed in (1) directly related, (2) 
somewhat related and (3) uru’elated jobs into a single number (AECD, 1996a) — fails to suggest and reward 
(and therefore enabling and motivating) corrective action. For example, technical-institute graduates are 
trained to perfonn particular jobs. The PBFM fails to differentiate between technical-focused institutions 
with high levels of employment in related fields (presumably due to appropriate training) and institutions 
with high levels of employment in unrelated fields (presumably due to inappropriate training combined 
with a shortage of general laborers) by aggregating all types of employment into a single number. At 
universities and colleges, the applicability of aggregated employment levels becomes more complex. The 
assessment of professional programs (e.g., engineering, law, commerce, etc.) is impacted in a manner 
similar to teclmical programs. A disaggregated indicator, however, would unfairly penalize liberal-arts 
programs, because they develop general skills that are applicable in a variety of situations. Whether a 
liistory graduate is employed doing (directly related) historical research or (unrelated) public relations is 
irrelevant: both positions were achieved based on superior research, analytical and ^vriting skills. This 
suggests tlie development of system-wide Pis has, in this case, led to a conceptually inadequate measure. 
The oversimplification of desirable outcomes is necessitated by the government's desire to tie funding to 
performance, which requires common performance measures — regardless of whether or not the measures 
are appropriate for all programs. 

Of furtlier concern is tlie assumption of scale linearity. Institutions increasing their employment rates from 
65% to 75% are rewarded \vith an additional 10 points — ^the same reward institutions receive for increasing 
employment rates from 85% to 95%. Obviously, the latter accomplishment is much more difficult 
(altliougli, as argued above, likely outside of institutional control) but tlie additional points garnered are the 
same. 

Examining Alberta's economy suggests that some areas may experience a negative correlation between 
employment rates of graduates and tlie enrollment levels that drive the accessibility indicator. Areas with 
strong.ties to tlie resource industry tend to experience a boom-and-bust cycle. During boom periods, 
employment rates of graduates may be close to 100% but attracting students is extremely difficult. The 
reverse is true during bust periods when tlie unemployed return to school to develop their skills for the next 





boom period. It seems unfair to structure a mechanism such that some institutions will find it difiFicult to 
score well on both of these indicators at once for entirely exogenous reasons. 

The graduate-satisfaction indicator also faces difficulties. Respondents asked their degree of overall 
satisfaction with tlieir program liave tliree choices: (1) fiilly satisfied, (2) somewhat satisfied, and (3) 
unsatisfied (AECD, 1996a). The satisfaction rating is the combined totals of (I) and (2). The results of this 
indicator are biased towards a favorable result by the construction of the scale: two of tliree choices 
(including the midpoint) are consider indicative of satisfaction. ‘Somewhat satisfied,’ logically, also 
indicates some dissatisfaction and combining it with ‘fiilly satisfied’ is misleading. 

Furtlier, only graduates are surveyed, thereby excluding the portion of each entering class that fail to 
complete a program. Tliis may result in sources of dissatisfaction going undetected because the root issue 
may serve to cull tliose affected prior to graduation. For example, programs that admit academically or 
socially unprepared students (who subsequently leave prior to completion) are failing to provide adequate 
levels of support to place-bound learners whose ability to seek educational opportunities elsewhere is often 
limited. The solution isn’t to compensate for this inadequacy by including an indicator measuring 
graduation rates (which itself is subject to a host of criticism (Cave et al., 1997)) but rather to design an 
indicator or process tliat determines tlie cause(s) of attrition and suggest what (if any) corrective action is 
required. Tliis recommendation assumes tliat educational policy is focused on maximizing access to 
programming rather than using post-secondaiy education as a mechanism to further sort people based upon 
previous social and/or educational status. 

The simple market model of satisfaction assumed by the PBFM is inadequate because education is an 
experiential good tliat entails the alteration of tlie consumer’s preferences during consumption (Cave et al., 
1997). If a consumer’s criteria forjudging a program have not fully (re)formed by the time satisfaction data 
is collected, tliey may not be in a position to definitively judge the quality of their educational experience 
(March, 1976). The assumption that graduates can accurately infer causal relationships between educational 
experiences and outcomes is dubious. Measuring satisfaction “assumes a simple logic of experiential 
learning: an action is taken; there is some response from the environment; there is some interpretation and 
evaluation of that response; and tlien a new action is taken reflecting the impact of the sequence” (March 
and Olsen, 1976, p. 56). If this cycle is broken (or if there is substantial interference such that the 
relationship between cause and effect is ambiguous) at any of the four steps, then the causal relationship 
tliat is inferred is to some degree deficient and the measure imperfect. 

Affordability indicators— Boi\i of the affordability indicators — enterprise revenue and administrative 
expenses — play a minor role tlie PBFM. Of note is the implicit focus on reducing the costs of post- 
secondary education borne by government and the exclusion of indicators tliat examine the affordability of 
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post-secondary education to students. If the indicators are designed to ensure institutions are providing 
quality learning opportunities to the greatest number of Albertans at a reasonable cost to learners and 
taxpayers, it would seem reasonable to expect some cost-effectiveness measures. There are no measures of 
the relationship between academic costs and quality learning opportunities. 

6. 2 Implications 

While it is difficult to isolate the impact of performance-based funding from the broader impact of 
academic capitalism, extrapolating from Alberta’s experience to date suggests PBFM has several 
implications: 

1. Declining autonomy — ^The growing use of performance and incentive envelopes shifts decision making 
power away from institutional governors and give it government (which determines the structure of 
funding envelopes) and corporations (who choose to partner with institutions based upon expected 
profitability). 

2. Grooving vocationalization — ^The Pis in the PBFM along with the criteria used to allocate other 
envelope funding pressures institutions to develop programs based upon labour-market demand. This 
has significant implications for programs in low demand or whose demand is difficult to quantify. 

3. Value-for-money approach to quality — Consistent with the focus on labour market outcomes is the 
notion of quality as tlie return on investment generated by programs. This ignores tliat liigher education 
has two roles: social reproduction and social criticism. This also obscures the qualitative impact of 
attempting to annually increase efficiency, such as growing class sizes, decaying infrastructure and 
deteriorating working conditions for faculty 

6.3 The future of performance-based funding in Alberta 

As noted above, tlie 1999 budget noted that the government will cease funding the Performance Envelope 
in 2000/01 although tlie value of performance awards will increase to 3% of institution’s operating budgets 
(AECD, 1999). Tliis means tliat the amount clawed back from each institutions will increase and, in turn, 
tliis would have a significant and immediate affect on institutions (particularly rural colleges which are 
precluded from performing well because of the boom-and-bust economy). Whether this will be politically 
unsaleable to rural MLAs remains uncertain. 

Also uncertain is tlie government’s ability and desire to maintain the PBFM system. The government has 
liad a great deal of difficulty gathering data on graduate employment and satisfaction rates (often using the 
same data for two or more years). Further, the government has had significant success at advancing its 
agenda (i.e., increasing efficiency and aligning institutional activities with the needs of the maiketplace) 
tlirough tlie use of incentive funding (i.e., funding envelopes). It may be that, ultimately, the accountabiUty 
component of using Pis will be retained but tlie regulatoiy aspect will not. 



7. Conclusion 

Alberta’s performance indicator-based accountability and funding mechanism is part of a broader policy 
agenda to cause and legitimate the introduction of academic capitalism. Academic capitalism is a response 
to the pressures of economic globalization and the rightist alliance. This agenda results in the alignment of 
higher education with tlie needs of the marketplace and reduces institutional autonomy. A number of 
criticisms of Alberta’s performance indicators and performance-based funding mechanism are evident. 
Unknown is the long-term sustainability of Alberta’s PBFM. 
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